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Abstract 

This article describes the design of a misconception diagnostic test about fire -related phenomena. It proposes a 
new test format in which a certainty-measuring tier has been integrated into each of the true/false response 
choices. This format is argued to be easier for teachers to use than the increasingly popular three-tier format. 
First, we review the available literature about misconception diagnostic tests and then literature about fire- 
related misconceptions. We then describe the design process of the test, which was preceded by an interview 
phase. We then describe its administration to 221 secondary school students. We finally present, in an 
explorative and accessible manner, the results that were obtained. These results support the existence of 
previously recorded misconceptions, but also bring certain nuances to some of their previous interpretations. 
They also support the hypothesis according to which some misconceptions are presumed to be more widespread 
than they truly are. Conclusive remarks are formulated about the benefits of the use of our — and other — 
misconception diagnostic tests. 
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Introduction 

The Use of Misconception Diagnostic Tests 

Successfully teaching for conceptual change in science first requires that teachers know about the existence and 
nature of possible “misconceptions” (DiSessa, 2006; Duit & Treagust, 2003; von Aufschnaiter & Rogge, 2010; 
Vosniadou, 1994) that could potentially interfere with learning. Misconceptions are ideas about how the world 
works that do not conform to scientific knowledge and are therefore considered by specialists as pedagogical 
obstacles. “The origin of these alternative conceptions may be grounded in traditional culture, knowledge, 
environment, economy, medicine, and personal thinking” (Chang, Lee, & Yen, 2010, p. 911). Thus, “identifying 
misconceptions and their causes prior to teaching becomes important in developing lessons that ultimately result 
in the reconceptualization of learning [...]” (Arslan, Cigdemoglu, & Modeley, 2012, p. 1668). To do so, 
teachers can get informed through science education literature or, if they have good reasons to believe that their 
particular students are not representative of previously studied learners (Chang et al., 2010; Stavy, Tsamir, 
Tirosh, Lin, & McRobbie, 2006), they can also use available diagnostic tests (Treagust, 1988). When these 
teachers become certain enough about the conceptual state of their students, they can begin preparing lessons 
that will challenge their misconceptions (Posner, Strike, Hewson, & Gertzog, 1982) or make more scientific 
conceptions prevail (Potvin, 2013) over the more naive ones. Thus, “assessment of misconceptions is very 
important for effective science instruction” (Pesman & Eryilmaz, 2010, p. 208). 

According to Caleon and Subramanian (2010), “the identification and investigation of alternative conceptions is 
one of the most important tasks in educational research” (p. 940). Therefore, it is not surprising that “over the 
past three decades, diagnostics tests have become a relatively prominent assessment tool in science education 
for data collection concerning the misconceptions on domain-specific knowledge” (Arslan et al., 2012, p. 1670). 

The most common type of test for this purpose is the multiple-choice (MC) test. This type of test is very 
interesting for diagnosing misconceptions because of the presence of distractors (incorrect yet tempting choices 
of answers) that correspond to frequent misconceptions. MC tests therefore allow for the assessment of not only 
assimilations (monotonic learning), but also accommodations, because when learners can not only produce a 
correct answer but furthermore resist attractive yet false ones, then we can suggest that their learning has gone 
beyond simple assimilation, or rote learning. Such well-designed tests sometimes allow researchers to see if 
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pedagogical treatments are enough to produce fundamental changes or not. In such an experiment, Gabel et al. 
used a multiple-choice test which allowed her to understand the strong persistence of misconceptions about 
burning though students were adequately exposed to scientific knowledge. The multiple-choice test, which was 
based on misconceptions reported in the literature, and the pre- and post-instruction interviews, indicated that 
“many misconceptions were persistent.” (Gabel, Monaghan, & MaKinster, 2001, p. 447) 


What Have We Learned From Recent Diagnostic Test Developments? 

In recent years, a number of new tests have been designed and proposed to researchers and teachers, and a new 
kind of format, the three-tier MC test, has progressively emerged. First, it is important to define what a second 
tier is: the most commonly used diagnostic test in science education is by far the classic Force Concept 
Inventory, which allows for the assessment of frequent misconceptions about simple mechanics. In 1995, the 
designers suggested that in order to improve its effectiveness, the test might benefit from a “second tier” that 
would ask students to justify their choice of answer in an open format. When justifying the use of such a second 
tier in their own test, Arslan et al. explained the following: 

Hestenes and Halloun (1995) recommended that a correct answer along with a wrong reason (false 
positive) and a wrong answer followed by a correct reason (false negative) be used to provide 
evidence for content validity [...]. Minimization of the probability of false negatives and false 
positives provides higher validity in multiple -choice tests. (Arslan et al., 2012, p. 1676) 

Pesman and Eryilmaz (2010) also reported the risk of false positives and false negatives in the design of their 
diagnostic test about electrical circuits. However, another risk was also reported, more inherent to the function 
of a diagnostic test. 

Although two-tier tests provide more information than other commonly applied methods for 
efficiently collecting data from large populations, some limitations have been identified. The 
presence of guessing may result due to overestimating the participants’ levels of knowledge as 
well as misconceptions as these tests do not discriminate lack of knowledge from misconceptions 
(Caleon & Subramanian, 2010; Pesman & Eryilmaz, 2010). An additional tier, which contains a 
certainty of response index, has been proposed to compensate for the likely weakness of the 
diagnostic tests (Hasan, Bagayoko, & Kelley, 1990; Pesman & Eryilmaz, 2010). (Arslan et al., 

2012, p. 1670) 

The Certainty of Response Index (CRI) is a confidence rating. This type of rating can “be regarded as an 
individual’s ‘internal, estimated belief’ in his or her own accuracy” (Renner & Renner, 2001, p. 23). Cordova et 
al. are also cautious in their description of the construct and add that “confidence in prior knowledge should not 
be confused with self-efficacy. Whereas self-efficacy in this context refers to a prospective judgment of one’s 
capabilities to learn about a specific topic, confidence in prior knowledge refers to a retrospective judgment of 
whether one’s current understanding of the topic is correct” (2014, p. 165). According to Pesman and Eryilmaz 
(2010, p. 209), Caleon and Subramanian (2010, p. 941), and Odom and Barrow (2007, p. 100), all of whom 
refer to Hasan et al. (1990), when confidence in prior knowledge is combined with an incorrect answer, this 
answer can be thought of as a strong clue to the presence of a misconception. Conversely, weak confidence 
expressed about a wrong answer can be considered as a mere lack of knowledge. “Distinguishing a lack of 
knowledge from a misconception is crucial because remediation of a lack of knowledge or a misconception may 
entail different instructional methods” (Pesman & Eryilmaz, 2010, p. 209). 

Although crucial, the pedagogical effects of high confidence in prior but false knowledge are unclear. Cordova 
reports cases in which it is presumed to strengthen prior knowledge and make change more difficult. But based 
on the available literature on feedback, she also reports cases in which certainty facilitates change. Her own 
results also point in this direction. She hypothesizes the following: 

It may be that when prior knowledge conflicts with the scientific information being presented, 
learners who are more confident in their prior knowledge may be more likely to pay attention to 
the alternative conception and overcome the barrier of inaccurate prior knowledge when specially 
designed conceptual-change based interventions, [. . .] are used. (Cordova et al., 2014, p. 172) 

According to Caleon and Subramanian (2010), “only a few researchers [...] have included confidence ratings in 
developing tests” (p. 941). Nevertheless, we believe it is very interesting because, if the “confidence” variable 
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can be integrated into research design, it could allow teachers to better design or choose the pedagogical 
treatments for certain profiles of certainty. In a recent research study, for example, we showed that traditional 
teaching settings might better benefit students in cases where they have doubts about their answers, and 
problem-based learning settings might better benefit students in cases where they feel more confident about their 
answers (Potvin, Riopel, Masson, & Fournier, 2010). 

However, to researchers’ despair, many of these tests, especially three-tier tests, though sometimes used by 
researchers, are very seldom used by teachers. These frequently overworked teaching professionals still appear 
to not be wholly convinced of the importance of diagnostics prior to teaching. Furthermore, the enormous task 
of administrating and analyzing these tests might be their coup de grace. We therefore believe that with the need 
for diagnostic, also comes the need for diagnostic tests that are easier to use. 


Students’ Conceptions about Fire 

Fire is one of the most common and familiar chemical reactions that students can witness (Chang et ak, 2010; 
Driver, Guesne, & Tiberghien, 1985). Burning is often used as an introductory example (yet often treated 
superficially) in chemistry textbook chapters (Boujaoude, 1991; Gabel et ak, 2001) at the secondary level. 
Nevertheless, it is subject to many misconceptions (Boo, 1995; Boujaoude, 1991), “making the study [...] of 
students’ understandings about this concept important for effective instruction” (Boujaoude, 1991, p. 691). 
Studies about fire have mostly used interviews, often supported by the observation of objects burning (wax, 
paper, alcohol, wood, etc.), as in studies by Bouajoude (1991) and Rahayu (1999). Some researchers have also 
used questionnaires, such as Watson (whose questionnaire, which is available, inspired us) or Prieto and Dillon 
(1995), and some have combined both (Gabel et ak, 2001). 

Recorded conceptions have been judged by researchers as being essentially “cued by visible aspects of events” 
(Boujaoude, 1991, p. 701). Rahayu (1999, p. 297) hypothesized that “combustion is a particularly difficult 
example of chemical change since it involves the gaseous state in both reactants and products, and 
decomposition as well as synthesis.” Finally, students’ conceptions are also presented as fragmented, 
inconsistent, and task specific. For example, “[...] most students described the burning of a candle as melting, 
the burning of alcohol as evaporation, and the burning of bread as changing color” (Boujaoude, 1991, p. 699). 

Inspired by Driver (1985), Boo reports a prototypical and rather simplistic idea of fire that many students 
seem to acknowledge as true: 

[...] burning involves a flame; products are smoke and an incombustible material known as the 
ash; the change is irreversible; and oxygen/air is needed, but its role is unclear or unknown. After 
specific instruction which includes the concept of interaction between oxygen and the combustible 
material, both studies reported that students are still unable to grasp the role of oxygen in 
combining chemically with the combustible material. Instead, they still show evidence of the use 
of perceptual thinking instead of conceptual thinking. (Boo, 1995, p. 52) 


Many other important invariants in students’ answers were recorded frequently enough to be considered as 
noteworthy conceptions by researchers and reviewers. Many of the most important ones are synthesized in 
Table 1. 
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Table 1. Some reported misconceptions about fire, combustion, or burning 


Misconception Author(s) 


• “The sole products of fire are smoke and an incombustible material 
known as the ash.” 

(Driver & Easley, 1985) 

• “Burning objects decrease in mass.” 

• “Metals are thought to melt or oxidise, but not burn.” 

• “Only carbon-containing compounds are thought to be capable of 
burning.” 

• “The candle wax in a burning candle is not burning, but only 
melting.” 

(Boo, 1995, pp. 53-54) 

• “Air is not actively involved in burning.” 

• “A substance is not used up, nor new substances formed in the 
burning process.” 

• “Heat is thought of as a substance that is formed in the burning 
process.” 

• “Particles can be destroyed in a flame.” 

(Scholium & Happs, 1982) 

• “Some combustibles are believed to be unable to burn, but only to 
melt or evaporate. For instance, combustion of a candle is 
interpreted as the wax melting, iron is said to melt, and the 
combustion of alcohol in an alcohol lamp is explained as alcohol 
turning into vapour.” 

• “Air is undergoing a transformation independent of the 
combustible material.” 

(Meheut, Saltiel, & Thibergien, 
1985) 

• “Oxygen or air is not involved in the change that is burning.” 

• “The flame or fire is an active agent of change.” 

• “Matter may be transmuted into heat.” 

• “One substance changes to a different form of the same substance.” 

• “Burning involves things going red and a flame appearing.” 

• “Solid residues or ash are the incombustible bits left behind.” 

(Watson et al., 1995, pp. 489- 
494) 

• “Alcohol and oxygen are not actively involved in burning.” 

• “Substances undergo no chemical reaction during burning.” 

• “Terms such as evaporation and burning can be used 
interchangeably when describing burning alcohol.” 

• “Phrases such as chemical change and physical change can be used 
interchangeably when describing burning things.” 

(Boujaoude, 1991, p. 700) 

• “The wick was the candle fuel.” 

(Gabel et al., 2001, p. 447) 


However, the set of misconceptions described in Table 1 does not seem to be internally consistent as a whole. 
Indeed, these misconceptions are not necessarily shared by the majority of students, or by students of the same 
age; some misconceptions seem to partially contradict others. Also, some of the statements hardly describe 
misconceptions, but rather conceptual errors (though they could serve as indirect clues to the presence of 
misconceptions). Nevertheless, they form a rather solid basis on which we decided to develop our own test (see 
below). 

We have also noticed in research that leads to the identification of misconceptions that the consideration of fire 
is often made under certain sets of curricular constraints. Sometimes, fire is seen as a security matter, or as a 
culturally interpreted phenomenon (Chang et al., 2010), but perhaps even more often as an essentially (almost 
exclusively) chemical phenomenon (Gabel et al., 2001). Indeed, subsections of studies include, for example, 
definitions, combustion, burning, products or conservation of matter — all chemically relevant angles that might 
not necessarily seem as central to young students as the colour of the flame, combustible (or not) materials, 
smoke, the necessary conditions to light a fire, or violent fires (explosions), etc. We will therefore choose not to 
limit ourselves to school-oriented perspectives on fire, but instead begin our investigation in a more open-ended 
mode, while still being attentive to what the research tradition has already identified. 
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Research Objectives 

Considering the elements presented above and the synthetic critique elements that emerged, we will focus on 
achieving two main goals: 

1) To develop through the knowledge of well-documented conceptions and open interviews an easy-to- 
use multiple-choice test that will allow teachers to diagnose the presence of misconceptions about fire. 

2) To conduct (with the developed test) an exploration of secondary students’ conceptions about fire. 


Method 

Design of the Instrument 

According to Pesman and Eryilmaz (2010, p. 208), “combining interviews and multiple-choice tests seems to 
be the best way for making sense of students’ understanding when taking the pros and cons of them into 
account.” We will therefore heed this advice by integrating both ways of gathering data. Also, according to 
Morales (2012, pp. 26-27), “high quality conceptual multiple-choice tests greatly differ from the traditional MC 
tests in the process of development [...]. The whole process involve[s] three major phases: (i) Preparation (in 
which reviews, observations and literature are considered); (ii) Development of the conceptual test [...]; and (iii) 
Validation.” The best three-tier tests were recently developed according to this process, with very few 
adaptations. Caleon and Subramanian (misconceptions about waves), for example, as well as Arslan 
(misconceptions about global warming, greenhouse effect, ozone layer depletion, and acid rain), were inspired 
by David Treagust’s (1988) recommendations, which suggested beginning with the identification (within an 
expert team) of concept boundaries; then questions “were written based on the concept boundaries and in 
accordance with the extensive related literature” (Arslan et ak, 2012, p. 1671-72); and finally the questions were 
piloted to an important number of subjects. Pesman and Eryilmaz (2010, p. 210) (misconceptions about simple 
electric circuits) also followed very similar phases. We therefore propose to rigorously follow this kind of three- 
step development process: Preparation, Development, and Testing. 


Preparation 

The authors (who all have science teaching experience as well as research experience) of this study piloted the 
preparation phase. Eor purposes of ease, it was decided from the beginning that we would renounce the use of 
the usual justification tier. We were aware that this might be at the expense of completely controlling the false 
negative and positive answers. But ease of use and the possibility of convincing teachers to use the test were 
judged to be sufficiently beneficial. However, it was decided that preserving the third tier (certainty in prior 
knowledge) could also contribute to reducing false negatives and positives, thus reducing the consequences of 
rejecting the second tier. 

Eollowing this, and inspired by previously recorded misconceptions about fire (see Table 1), an interview grid 
was created in order to assist the interviewers in collecting coherent data as well as encouraging fruitful 
interviews. This guide was given to 24 interviewers (who all have bachelor degrees in science), divided into 
teams of 2 to 4 interviewers, with the mandate to to ask open-questions to 7 secondary school students (per 
team). In this exploratory phase, we used the services of many interviewers in order to favor the emergence of a 
fair variety of misconceptions. A total of 42 voluntary secondary-school students with no special training about 
fire (except ordinary school curricula) were interviewed for durations of half an hour to an hour and voice- 
recorded. Interviewers received prior training on the importance of providing a good interview climate, of 
welcoming verbalizations without judgement or feedback (except encouragement to talk), of avoiding questions 
that begin with the word “why” (judged as too school-like and typically used for testing conceptions instead of 
exploring them), of favouring Ericksonian reflection (repeating the subjects’ last words to encourage them to 
deepen their explanations), of occasionally asking students (in an optional fashion) about their level of certainty 
(1 to 5), and of thanking them for their participation (at the end), etc. The interview guide they used proposed 
the following start-up questions: 

• What is fire? (with the possible use of a photo of a wooden “camp-style” fire [see Eigure 1]) Other 
possible “start-up” words: flame, ember, ash, smoke, crackling, burning brand. 
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• How does it work? Other possible supporting questions: Where does the heat come from? Where does 
the wood go? Why do the flames go up? What’s left of the wood after it burns? What do you need to 
light afire? What can extinguish afire? How can fire be extinguished? 

• To conclude: What else do you know about fire that you would like to explain to me? 



Figure 1 . Image of a wood fire 


Interviewers were also told to ask more precise questions if they had time or to fully pursue interesting lines of 
thought, when applicable. 

After the interviews, the interviewers submitted multi-page written reports synthetically describing the recorded 
verbalizations. The interviewers and the three experts also held a two-hour workshop together to share the 
interesting conceptions that they had inferred and recorded in their reports. 


Development 

Based on the above-mentioned preparation phase and on the collected data (literature and interviews), the 
development of the questionnaire began. It immediately appeared to the expert team that many of the potentially 
interesting ideas could hardly be gathered into four or five choices of questions. Instead, it was suggested and 
decided that every “miseoneeptual” response could easily be translated into a true/false format. Many of these 
questions could, however, be gathered into sections of a questionnaire (for example, a section about “What can 
bum?”). In retrospect, this choice was also interesting for another reason: MC always suggests that there is only 
one correct answer. However, it is not implausible that a subject would adhere to more than one distractor. Thus, 
our decomposition of each possible MC question into multiple true/false questions would allow for a subject to 
fall for more than one distractor, in turn allowing the recording of multiple misconceptions. Indeed, distractors, 
unlike correct answers, are not necessarily mutually exclusive. 

With this decision, it appeared that it would also be possible to integrate the third tier (certainty) with the first 
one, providing a very simple -to-handle (and simple-to-analyze) format. For example, every question had the 
same format as the example in Table 2: 


Table 2. An example of a question and the possible choices of answers given to students 


Question 

(statement) 



Choices 



Plants can burn. 

I’m certain it is true. 

I suspect it is true. 

I do not know. 

I suspect it is false. 

I’m certain it is 
false. 


The team included questions that tested all the misconceptions they suspected to be frequent. They were 
suspected to be frequent when more than one interviewer team recorded similar incorrect answers from different 
students. At that point, the questionnaire went into a two-week iterative development phase, during which time 
discussions based on reports were conducted among the experts on a daily basis. At the end, a 79-question 
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document (divided into 13 theme-based sections) was proposed for testing. A discussion was held about the 
correct answer to attribute to each question, and research of scientific documents was carried out when an 
agreement could not be reached. When an agreement still could not be reached about the correct answer even 
though research of scientific documents had been conducted, the question was reformulated until the team was 
satisfied, or else it was deleted. 


Testing the Instrument: Exploring Conceptions and Misconceptions 

Sample and Procedure 

The subjects were 212 French-speaking students from secondary schools in the greater Montreal area, Canada. 
They were recruited by their teachers, who were asked to cooperate with the project by the researchers. There 
were 134 girls and 78 boys (the majority of girls is due to recruitment at a few girls-only schools). The vast 
majority were in grades 9 (n=94) and 10 (n=94), with very few from grades 8 (n=9) and 1 1 (n=8). The mean age 
was 15 years and 8 months with a standard deviation (SD) of 1 year (363 days). The portrait is therefore rather 
representative of students from the middle of secondary school (although a bit more of girls than boys). 

Students were read the instructions by the teacher and were invited to complete the questionnaire by answering 
to the best of their ability. They had half an hour to do so. They responded to three socio-contextual questions 
(girl/boy, birthdate, school level) and then to 79 conceptual questions. They were told that even though the 
statements were in groups, they had to choose one of the five choices (I’m sure it is true; I suspect it is true; I 
don’t know; I suspect it is false; I’m sure it is false) for each of the statements. 

Afterward, we first checked for cases who selected the same answer for all questions, marked the answer in any 
order like Z shape or any repeated orders (lack of honesty) were dropped from the study, and then proceeded to 
input and analyze the data. We wanted this analysis to be as simple as possible in order to convince teachers that 
they did not need advanced knowledge in statistics to use the test and be able to interpret the data. Other classic 
validations of tests use factor analysis, internal consistency indexes (Cronbach alphas) and other means, but we 
chose to work essentially on descriptive statistics. This way of working was also better suited to an exploratory 
study and furthermore while using a questionnaire containing strictly ordinal data. 


Results and Discussion 

The results were presented under three categories: the first is about potential “misconceptual” answers, which 
will include answers that reached a threshold of (arbitrarily decided) 33% general inaccuracy (initially 
regardless of expressed certainty); the second is about potential “lack of knowledge” answers (at least 33% of 
“do not know” or missing answers); and the third is about potential “accurate” answers (at least 33% general 
accuracy), which will be presented as perhaps too easy to be included in future versions of our diagnostic 
questionnaire. 

Potential “Misconceptual" Answers 

For purposes of uniformity and to facilitate the interpretation of tables 3, 4, and 5, we have corrected the 
answers and presented them in “accurate/inaccurate” instead of true/false format, which would necessitate 
repeatedly going back and forth to see the correct answer every time. Therefore, for the following tables, the 
coloured columns show a left bright “inaccurate to accurate” continuum, with the most certain answers at the 
ends. 

Table 3 gives the percentages of answers for the 20 most “inaccurate but certain” statements among the ones 
which got at least 33% total inaccuracy. Statements are presented in decreasing order of “inaccurate but certain” 
percentages of answers. For an equal number of these answers, a decreasing order of “inaccurate but uncertain” 
answers prevails. Missing answers are not included in the percentages. 
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Table 3. Distribution percentage of potential “misconceptual” answers for each statement 


Statement 


The Sun is a true fire. 

Diamonds can burn. 

Flames go up because they search for oxygen, which is in the air 
and not in the soil. 

Fire is a gas. 

Lava is a true fire. 

Lighting a match is absolutely necessary to start a wood fire. 
Fireworks are a true fire. 

The tip of the flame is hotter than the base. 

A bigger fire is always hotter. 

Water can extinguish a fire because it neutralizes the flames. 
Friction (like rubbing two rocks together) is absolutely 
necessary to start a wood fire. 

At its origin, light from fire came from energy that came from 
the Sun. 

Smoke is a part of the cycle of fire. 

Fire is made of matter. 

Black smoke (instead of whiter smoke) indicates that the fire is 
more intense. 

The ember is hotter than the flame. 

Water can extinguish a fire because it transforms fire into 
smoke. 

Metals can burn. 

Smoke contains oxygen. 

Carbon dioxide (CO 2 ) is absolutely necessary to start a wood 
fire. 

Smoke is partially made of steam. 



We believe that the first 10 questions provide valuable hints that possibly betray the presence of 
misconceptions. Not only do they have a lot more “inaccurate but certain” answers than all the other types of 
answers, but they also show more “inaccurate but certain” answers than “inaccurate and uncertain” answers, 
indicating that many students probably hold deeply rooted misconceptions on which they base their answers. 
Indeed, “by definition, misconceptions are strongly held cognitive structures that are not consistent with 
scientific concepts (Hammer, 1996). This definition can be restated as follows; for a conception held by a 
student to be a misconception, it requires not only being inconsistent with scientific concepts but also being 
strongly advocated by the student” (Pesman & Eryilmaz, 2010, p. 209). 


Some of the misconceptions are rather surprising. Of course, we already knew that most students have difficulty 
recognizing that metals or wax can burn (see question 2, hereafter designated [like all other questions] as Q2: 
24% [all percentages presented in this section are for “inaccurate but certain ” responses]), but the scores for 
diamonds (Q3: 50%) are much stronger. It might be that students confuse the well-known mechanical strength 
associated with diamonds with other “durability” properties. Students also seem to have significant difficulty 
determining what fire truly is. We already knew that students had trouble recognizing fire as an event (a 
chemical reaction), but they also seem to agree that it is made of matter (Q39: 26%), more precisely a gas (Q45). 
This impressive score (45%) could, however, be an effect of the structure of the questionnaire because it asked 
in a sequence if fire was a liquid, a solid, or a gas (Q43-Q45). Perhaps students felt the need to choose between 
at least one of these three possibilities. However, the association of fire with the idea of gas has already been 
recorded. 


Apparently students do not feel strong discomfort identifying the Sun (Q12: a strong 60%) and lava (Q1 1; 40%) 
as true fires (even though there is obviously no air in space and lava does not produce flames [it is, however, 
incandescent]). It might be that very high temperatures that produce incandescence are hard to accept as not 
being fires. We could also report that common socially shared expressions such as “solar fire” can interfere with 
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school messages. However, fireworks seem to be more difficult to qualify, as they should be. We could 
hypothesize that the brevity of the fire phenomenon does not contribute to its qualification. The temperature of 
fire also seems to be problematic. Students seem to adhere to the idea that the centre (or base) of a fire is hotter 
than the rest, even hotter than the tip of the flame (which is not the case, however, with Q77: 38%); likewise, an 
ember is considered to be hotter than the flame (Q76: 26%), which (surprisingly for some of us) it is not. Bigger 
fires are also considered to be necessarily hotter than smaller ones (Q74; 36%) and to produce darker smoke 
(Q60; 26%), which is not necessarily the case. It also seems to be difficult for students to see the flame as a 
product of fire. Instead, they seem to attribute an active role to the flame; the statement that “flames go up 
because they search for oxygen, which is in the air and not in the soil” got an impressive 48% approval (+ 
certainty). In Q71 (35%), students agree that “water can extinguish a fire because it neutralizes the flames.” 
These last observations are in line, we believe, with Watson’s observation that the flame is an active agent of 
fire (Watson et ah, 1995, pp. 489-494). However, we should note that it is true that when flames are absent, the 
fire should be considered as extinguished, by definition. 

However, we believe that the rather strong score of 39% obtained for Q57, “lighting a match is absolutely 
necessary to start a wood fire,” should not be considered as strong as the other “misconceptual” answers. 
Indeed, it also got a strong 41% of “accurate and certain” answers. We therefore believe that different students 
might have understood the question differently. Thus, we do not recommend including it in future versions of 
the questionnaire. We also noted similar though weaker effects for Q48 and Q53, which are similarly 
constructed. It is, however, a little surprising for the latter that some students would strongly see carbon dioxide 
as necessary to start a wood fire. It is not impossible that students know it is involved in the process, but are 
confused about its role. This has also been recorded in the past for oxygen (Scholium & Happs, 1982; Watson et 
ah, 1995). 

Another rather strong result is that students do not seem to acknowledge that “at its origin, light from fire came 
from energy that came from the Sun” (through photosynthesis) (Q23: 30%). However, it is not easy to 
determine which misconception could be at the origin of this wrong answer. It is possible that the question 
simply appeared odd (and therefore difficult to agree with) because of a lack of knowledge about the origin of 
the energy that is released through, for instance, a wood or gasoline fire. 

Other strong results for “inaccurate but certain” answers could reveal misconceptions, but since “inaccurate but 
uncertain” answers become more popular as we descend the list (even becoming stronger than “inaccurate but 
certain” ones), we believe it is not safe to think of them as revealing strong misconceptions and fulfilling 
Hammer’s strict criteria. However, we believe that for the “misconceptual” answers discussed above, teaching 
might benefit more from accommodation initiatives than from regular teaching methods. 


“Do Not Know ’’ or Missing Answers 


Table 4 provides the distribution percentages of answers for statements that got at least 33% total “does not 
know” or missing. It is a little hazardous to associate missing answers to “lack of knowledge”; therefore, 
statements are presented only in decreasing order of “do not know” number of answers. For an equal number of 
these answers, a decreasing order of “missing” answers prevails. 


O 

6 

34 

70 

76 


Table 4. Distribution percentage of potential “lack of knowledge” answers for each statement 


Statement 


Asbestos can burn. 

The colour of a flame depends on magnetic fields. 
Water can extinguish a fire because it changes the 
nature of the combustible. 

The ember is hotter than the flame. 



Few questions (only four) qualify for this category. This interesting result, especially considering the fact that so 
many answers in the test were inaccurate, could be considered as a result in and of itself. Indeed, Cordova has 
already argued that “students still generally tend to be overconfident” (Cordova et ah, 2014, p. 165). The small 
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size of Table 4 (in comparison with tables 3 and 5 [see below]), we believe, supports this claim. Students often 
do not dare to reveal their lack of knowledge. They clearly preferred to provide positive or negative answers, 
even if they were uncertain. 

We believe the most surprising result is Q6 about asbestos. Living in the province where the most asbestos in 
the world is produced, the students still did not seem to know (45%) or were wrong about (38%) its flameproof 
properties. Clearly, Quebec schools should do more to teach students about asbestos and its history. 

Results for Q34 are not surprising, since it is plausible that most students never had the opportunity to establish 
links between fire and magnetic fields. We could, in fact, consider the 53% score of students who got the answer 
right as encouraging. The same could be said about question Q70, that “water can extinguish a fire because it 
changes the nature of the combustible.” We already discussed Q76, that “the ember is hotter than the flame,” 
because it also got rather high inaccuracy scores. This rather puzzling result might suggest that for this question, 
students are generally either deceived, or not informed. 


“Accurate ” Answers 


Table 5 provides the percentages of answers for the 20 most “accurate and certain” statements among the ones 
which got at least 33% total accuracy. The statements are presented in decreasing order of “accurate and 
certain” percentages of answers. For an equal number of these answers, a decreasing order of “accurate but 
uncertain” answers prevails. Missing answers are not included in the percentages. 


0 

47 

55 

52 

15 

50 

43 
46 

51 

44 
27 

1 

9 

18 

40 

56 
7 

10 

49 

63 

41 


Table 5. Distribution percentages of answers for statements that are potentially too easy 


Statement 


A combustible (e.g., wood) is absolutely necessary to 
start a wood fire. 

The colour red is absolutely necessary to start a wood 
fire. 

The presence of the Moon is absolutely necessary to 
start a wood fire. 

A lit candle is a true fire. 

A combustive (e.g., oxygen) is absolutely necessary to 
start a wood fire. 

Fire is a liquid. 

Air is absolutely necessary to start a wood fire. 

The presence of the Sun is absolutely necessary to start 
a wood fire. 

Fire is a solid. 

Ash is a product of the combustion of wood. 

Plants can burn. 

Certain gases can burn. 

Flames go up because they are attracted by the Sun’s 
gravity. 

Fire is a chemical reaction. 

Wind is absolutely necessary to start a wood fire. 
Certain solids can burn. 

The flame of a barbecue is a true fire. 

A magnifying glass is absolutely necessary to start a 
wood fire. 

Smoke contains carbon dioxide (CO 2 ). 

Fire is a mechanical reaction. 


T 

F 

F 

T 

T 

F 

T 

F 

F 

T 

T 

T 

F 

T 

F 

T 

T 

F 

T 

F 
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Of course, these last items that did not allow us to record any misconceptions. The reader might be a little 
puzzled as to why some of these questions were included in the questionnaire. The reason is that we 
systematically chose to include questions that appeared at least more than once in the interviews in order to 
encourage the possibility of identifying new and unexpected misconceptions. Obviously, certain results show 
that some of them were anecdotal. 

Since some accuracy rates are so high, it is difficult to consider that they should be given more attention in class. 
However, sometimes, an imperfect accuracy rate could suggest that more assimilation efforts could be made. 
But, for a diagnostic test like the one we are developing (with a rather strict set of criterion for identifying 
misconceptions), they are not the most useful. However, rejecting them all is not necessarily advisable, because 
if every question left creates tension within the subjects, it could alter their perception of the exercise and unduly 
increase their vigilance, causing their answers to be less spontaneous. 

However, we noticed that some of these high-scoring questions are somewhat surprising. For example, the 
statements of Q26, Q40, Q50, and Q63 appear, despite their technical character, to be widely accepted. This 
should be credited to the schools. Students seem to easily acknowledge that fire is a chemical reaction that needs 
a combustible material and which produces carbon dioxide. We can also say the same about the answers to Q68, 
Q69, and Q73 (which are about the mechanisms by which one can extinguish a fire and which all got above 
52% accuracy, despite the fact that they can be judged as rather misleading). Q38 (“The colour of the flame 
depends of the quality of the blend with oxygen”) and Q17 (“Flames go up because the air that’s heated by the 
fire is lighter than the surrounding air [for comparable volumes]”), which are quite technical, still got a positive 
62% for general accuracy. 


Other Interesting Percentage Distributions 

The three preceding tables (3, 4, and 5) show the answer distributions that are interesting in light of 
misconceptions, lack of knowledge, and (excessive) easiness. But there are also other distributions of certain 
questions that are interesting for our discussion. Most of them occur in the continuity of Table 5 (which was 
limited to the 20 highest accuracy scores). They show high accuracy scores, even though theses scores might be 
somewhat surprising in light of the available literature or the interview phase of our research. Indeed, some have 
been discussed at length by the authors or have appeared repeatedly in the interviews. We therefore could have 
formulated positive hypotheses about their frequent inaccuracy. But even though accuracy/inaccuracy does not 
have absolute thresholds, our team was nonetheless surprised to see them get such high accuracy scores. 

• Q4: “Certain minerals can bum” (true) got 54% accuracy, despite the fact that literature shows that 
students often view minerals as incombustible (Boo, 1995; Meheut et al., 1985). 

• Q28: “Ash is made only of wood bark, which does not bum” (false) got 61% accuracy, despite the fact that 
a certain number of students said in the interviews that bark does not bum. 

• Q67: “Water can extinguish a fire because it is stronger than fire” (false) got 51% accuracy, even though it 
came up frequently in the interviews. 

• Q66: “Smoke is a part of the clouds” (false) shows that there is a rather good distinction made between 
clouds and smoke, despite the fact that it has been presented as a teaching difficulty (Potvin, 2011). 

• Q42: “Fire is a phase change” (false) got 42% accuracy, even though literature shows that students often 
see combustion as such (Boo, 1995; Meheut et al., 1985). In the same line of thought, Q31 (“Ash is 
chemically different from wood” [tme]) got an encouraging 58% accuracy, with as much as 32% of 
“accurate and certain” answers). 

• We also already knew from Gabel’s work that students often think that certain liquids cannot burn (Gabel 
et al., 2001). It is therefore a little surprising to see that Q8 (“Certain liquids can burn”) got 45% “accurate 
and certain” answers. 

We believe these interpretations are interesting because they challenge the idea that such conceptions are widely 
spread. In every single case described above (except the first one), the “accurate and certain” scores were even 
stronger than the “accurate but uncertain” ones, which seems to strengthen the claim that students did not adhere 
strongly to the associated misconceptions. 

Indeed, during teaching, it is not possible to focus on all possible conceptions. Therefore, only the most frequent 
misconceptions should be addressed systematically, whereas others may be treated on a more individual basis. 
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Toward a New Version of the Test 

In light of the precedent analysis of the data, we suggest that further development of our diagnostic test should 
reject or reformulate certain questions for clarity and economy. Such development should, in our opinion, take 
into account the detailed and coded suggestions that appear in Appendix 1 . However, it might be inadvisable to 
reject all “too easy” questions, because it could probably cause “whole test effect” differences, changing the 
attitude of the students toward the test. Also, some too easy questions could allow teachers to better understand 
that certain misconceptions might not be as widespread as anticipated (as discussed above). 

We hope that teachers will be inclined to use the test or certain questions in order to design their own. We hope 
that some will choose to strategically administer the test to their students and use the results to monitor their 
progress. We also hope that they share their results with us for comparison. 


Conclusion and Recommendations 

In this article, we described the development and the first actual test of a conceptual diagnostic instrument about 
fire. We did not proceed with a typical validation of the test, but instead concentrated our efforts on creating an 
instrument that would be easy to use and analyze. Therefore, a true/false format was proposed, with a certainty 
tier integrated in the answer choice. The test was inspired by the available literature about conceptions 
concerning fire, combustion, and burning, but also by an initial open-ended interview phase. It allowed the 
researchers to confirm the presence of answers that could suggest the presence of certain classic misconceptions 
about fire, some of which had already been recorded in literature, such as the conception that hard solids, like 
metals or diamonds, cannot burn, and difficulties qualifying fire as such. Indeed, according to our results, 60% 
of all subjects, for example, believed that the Sun is a true fire. Our results also seem to confirm an “active” role 
of the flames, which are not conceived as an emanation. We believe that the “certainty” tier of our test allowed 
us to confirm the solidity of certain misconceptions and also allowed us to — at least partially — reduce 
interpretations that could have been the product of guessing or “on the spot” context-based intuitions. Indeed, 
we chose to qualify as misconceptions only those responses with a strong “inaccurate but certain” distribution. 
Regarding identified misconceptions, we would like to warn teachers that it is more than possible that they will 
encounter them while teaching and we suggest that accommodation teaching initiatives be undertaken instead of 
mere assimilation methods. 

We also believe that our results show that students are often overconfident about their spontaneous reasoning. 
Indeed, many answers showed high inaccuracy levels even though the possibility of answering “I do not know” 
without any consequence was given. We believe that teachers should be made aware of this and that students are 
often not humble enough when it comes to a familiar phenomenon like fire. We therefore believe that 
“transmissive” teaching models that presume such modesty might not be appropriate as often as thought. It is 
also possible, however, that students do not develop enough of a humble attitude because school usually does 
not encourage it. Instead, answers, good or bad, are always required in order to test accurately, and uncertainty 
is not an encouraged attitude. Most of the time, we believe that it is seen as a weakness. Students might have 
transferred such a “misconception” about uncertainty while answering our test questions. 

Finally, we believe that our instrument provided the opportunity to test the presumed widespread character of 
some previously recorded misconceptions. For example, the statement “certain minerals can bum” surprisingly 
recorded a rather fair accuracy score (“metals” did not fare so well and “diamonds” were second worst). Q42 
(“Fire is a phase change”) also got more accuracy than inaccuracy, which challenges the importance given to a 
confusion between physical/chemical changes by previous literature (Boujaoude, 1991). 

We believe that our questionnaire could allow teachers to do the same: not only test for the presence of 
misconceptions, but also “put to the test” how solid they are. We think these two last ambitions are possibly as 
legitimate as the first one. We also believe the contribution of our research might lead in this direction: to raise 
awareness among teachers about misconceptions but, furthermore (and perhaps more importantly), to help them 
get in line with the conceptual state of their class (without overdoing it) and plan effective and economical ways 
to promote scientific accommodations as well as assimilations. 

Our test unfortunately might not have the robustness of other well-designed diagnostic tests that were previously 
proposed and validated by strong research designs (Arslan et ah, 2012; Caleon & Subramanian, 2010; Morales, 
2012; Pesman & Eryilmaz, 2010; Treagust, 1988). For example, for now it remains at an exploratory phase and 
it also rejected the second tier of investigation, which was strongly argued as important in previous literature. It 
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is therefore possible that our test recorded false negatives and false positives in more than tolerable quantities. 
We are fully aware of this possibility, but we also believe that to be useful, tests must first be used. This is why 
we proposed a format that avoids heaviness in management. Thus, we invite all possible users of the test to 
contact us and tell us what they think of it and its shortcomings. We will be more than happy to propose 
improved versions based on such exchanges. 
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Appendix A 

The 79 statements of the questionnaire and the proposed recommendation for each for future versions of the test 
are presented below in Table 6. Legend: For rejection (E) means too easy, (S) means potential structural 
problem (identified in retrospect), (NR) means not necessarily relevant for a diagnostic test considering the 
final answers. 


Table 6. Question number, associated statements, correct responses, and recommendations 


Question 

number 

Statement 

Correct 

response 

Recommendation 
after analysis 

1 

Plants can burn. 

T 

Reject (E) 

2 

Metals can burn. 

T 

Retain 

3 

Diamonds can burn. 

T 

Retain 

4 

Certain minerals can burn. 

T 

Reject (E) 

5 

Concrete can burn. 

F 

Reject (E) 

6 

Asbestos can burn. 

F 

Retain 

7 

Certain solids can burn. 

T 

Reject (E) 

8 

Certain liquids can burn. 

F 

Retain 

9 

Certain gases can burn. 

T 

Reject (E) 

10 

The flame of a barbecue is a true fire. 

T 

Reject (E) 

11 

Lava is a true fire. 

F 

Retain 

12 

The Sun is a true fire. 

F 

Retain 

13 

Fireworks are a true fire. 

T 

Retain 

14 

The filament of a light bulb is a true fire. 

F 

Reject (E) 

15 

A lit candle is a true fire. 

T 

Reject (E) 

16 

Flames go up to flee the heat that is at the base of the fire. 

F 

Reject (E) 

17 

Flames go up because the air that’s heated by the fire is lighter 
than the surrounding air (for comparable volumes). 

T 

Reject (E) 

18 

Flames go up because they are attracted by the Sun’s gravity. 

F 

Reject (E) 

19 

Flames go up because they follow electric charges in the air. 

F 

Reject (NR) 

20 

Flames go up because they search for oxygen, which is in the air 
and not in the soil. 

F 

Retain 

21 

Light comes from the heat of the fire. 

T 

Reject (S) 

22 

The burning wood transforms directly into light. 

F 

Reject (E) 

23 

At its origin, light from fire came from energy that came from the 
Sun. 

T 

Retain 

24 

Fire does not emit light; it absorbs light from its surroundings. 

F 

Reject (E) 

25 

Fire is not made of matter, thus it absorbs light. 

F 

Reject (E) 

26 

Light comes from the transformation of chemical energy into 
light energy. 

T 

Reject (E) 

27 

Ash is a product of the combustion of wood. 

T 

Reject (E) 

28 

Ash is made only of wood bark, which does not burn. 

F 

Reject (E) 

29 

Ash is made only of incompletely burnt wood. 

F 

Reject (E) 

30 

Ash has the same mass as burnt wood (1,000 g of wood will 
produce 1,000 g of ash). 

F 

Reject (E) 

31 

Ash is chemically different from wood. 

T 

Reject (E) 

32 

Ash comes from something other than wood, since it has 
completely burned. 

F 

Reject (E) 

33 

Ash is a part of smoke. 

T 

Reject (E) 

34 

The colour of a flame depends on magnetic fields. 

F 

Reject (NR) 

35 

The colour of a flame depends on the presence of certain 
combustibles. 

T 

Reject (E) 

36 

The colour of a flame depends on the altitude of the fire. 

F 

Reject (E) 

37 

The colour of a flame depends on the temperature of the 
surroundings. 

F 

Reject (E) 

38 

The colour of a flame depends on the quality of the blend with 
oxygen. 

T 

Reject (E) 
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39 

Fire is made of matter. 

F 

Retain 

40 

Fire is a chemical reaction. 

T 

Reject (E) 

41 

Fire is a mechanical reaction. 

F 

Reject (E) 

42 

Fire is a phase change. 

F 

Reject (E) 

43 

Fire is a liquid. 

F 

Reject (E) 

44 

Fire is a solid. 

F 

Reject (E) 

45 

Fire is a gas. 

F 

Retain 

46 

Air is absolutely necessary to start a wood fire. 

T 

Reject (E) 

47 

A combustible (e.g., wood) is absolutely necessary to start a 
wood fire. 

T 

Reject (E) 

48 

Friction (like rubbing two rocks together) is absolutely necessary 
to start a wood fire. 

F 

Reject (S) 

49 

A magnifying glass is absolutely necessary to start a wood fire. 

F 

Reject (E) 

50 

A combustive (e.g., oxygen) is absolutely necessary to start a 
wood fire. 

T 

Reject (E) 

51 

The presence of the Sun is absolutely necessary to start a wood 
fire. 

F 

Reject (E) 

52 

The presence of the Moon is absolutely necessary to start a wood 
fire. 

F 

Reject (E) 

53 

Carbon dioxide (CO 2 ) is absolutely necessary to start a wood fire. 

F 

Reject (S) 

54 

A sufficient quantity of heat is absolutely necessary to start a 
wood fire. 

T 

Reject (E) 

55 

The colour red is absolutely necessary to start a wood fire. 

F 

Reject (E) 

56 

Wind is absolutely necessary to start a wood fire. 

F 

Reject (E) 

57 

Lighting a match is absolutely necessary to start a wood fire. 

F 

Reject (S) 

58 

Smoke is created by the wind. 

F 

Reject (E) 

59 

Smoke is partially made of steam. 

T 

Reject (E) 

60 

Black smoke (instead of whiter smoke) indicates that the fire is 
more intense. 

F 

Retain 

61 

Smoke contains ash. 

T 

Reject (E) 

62 

Smoke is a part of the cycle of fire. 

F 

Retain 

63 

Smoke contains carbon dioxide (CO 2 ). 

T 

Reject (E) 

64 

Smoke can change colour depending on what is burnt. 

T 

Reject (E) 

65 

Smoke contains oxygen. 

F 

Retain 

66 

Smoke is a part of the clouds. 

F 

Reject (E) 

67 

Water can extinguish a fire because it is stronger than fire. 

F 

Reject (E) 

68 

Water can extinguish a fire because it prevents photosynthesis in 
the wood. 

F 

Reject (E) 

69 

Water can extinguish a fire because it prevents oxygen from 
reaching the fire. 

T 

Reject (E) 

70 

Water can extinguish a fire because it changes the nature of the 
combustible 

F 

Reject (NR) 

71 

Water can extinguish a fire because it neutralizes the flames. 

F 

Retain 

72 

Water can extinguish a fire because it transforms fire into smoke. 

F 

Possibly retain 

73 

Water can extinguish a fire because it cools fire. 

T 

Reject (E) 

74 

A bigger fire is always hotter. 

F 

Retain 

75 

When a fire gets hotter, it changes colour. 

T 

Possibly retain 

76 

The ember is hotter than the flame. 

F 

Retain 

77 

The tip of the flame is hotter than the base. 

T 

Retain 

78 

The heat of fire is generated by its light. 

F 

Reject (E) 

79 

The heat of fire depends on the combustible material. 

T 

Reject (E) 























